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A formative evaluation of a University of Tennessee, 
Knoxville, predoctoral project was conducted. The program, which was 
funded by the federal Minorities and Women's Program, was designed to 
increase the participation of minorities and women in educational 
research and development. The project, "Experimental Program for 
Opportunities in Advanced Study and Research in Education," involved 
18 minority and female second-year doctoral students, 19 faculty 
members, and 8 individuals holding doctorates and employed in local 
education agencies or other organizations. Students who received 
fellowship support believed that the project experience had been most 
effective in increasing their abilities to engage in an internship, 
work with a mentor, strengthen network ties, and write a research 
report. According to participants, other factors that distinguished 
c?> this program from a typical doctoral program included 

interdisciplinary contact, support for research services and travel, 
and exposure to a range of issues of interest to minorities and 
women. A majority of participants believed the project experiences 
enhanced the employability of students. Several measures indicated 
/that both the production and utilization of research increased as a 
result of the project. (SW) 
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Background 

The Minorities and Women's Program of the National Institute of 
Education was established in 1977 in response to a resolution of the 
policy-making body for the NIE, the National Council of Educational 
Research, which declared that: 

"It shall be the policy of the National Institute of Education 
to increase the participation of minority persons .and women 
in research and development efforts of the nation through 
ensuring that qualified minority firms and individuals and 
qualified women are given informed opportunity >.feo participate 
in NIE programs, and through efforts tp increase the numbers, 
qualifications and performance of minority firms and , 
individuals, and women engaged in R&D (Baker, 1979, p. 3)." 

The principal objective of the Minorities and Women's Program has beetl 
to increase the quality and relevance of educational research and develop- 
ment by seeking ways to a) assist minority and women scholars to participate 
at advanced levels of educational research and related work, and b) overcome 
barriers* to recognition of research contributions, perspectives, and 
interests of minority persons and women (Baker, 1979, p. 9). 

This objective is predicated on the well-documented conclusion that 
the social context of education is adversely affected by the underrepre- 
sentation of minorities and women in educational research and development • 

Forbes has observed that 

"Most ERD (educational research and development) work then has 
been carried out by people who are, by definition, not involved 

, in the world dealt with by the research . . . Thus we discover 
not only ethnic bias and elitism but also the difference 
between a theoretical problem conceived by a "detached" 
researcher . . . and a problem arising in the actual process 
of education and brought to the level of R and D by actual 
practitioners or target groups . . . the White theoretician 
R and D specialist discovers "a problem" which is not the same 
"problem" seen $y a non-White practitioner or consumer (1977, 
p. 4)." 

According to Martin, "when the activities and experiences of females 
arc excluded from the educational realm, those of males provide our norms 
(1982, p. 147)." 
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Baker has added that 

"R&D agendas and methodologies by-pass or are ill-suited to 
address the stubborn problems plaguing urban and rural schools 
attended by poor children and by minority children. Then 
too, staffing patterns in educational research and educa- 
tional institutions undermine the relevance, focus, and 
credibility of improvement efforts, since they are led by 
persons from outside the populations intended to be served 
(1979, p. 4)." ... 

Finally , Bagenstos suggests that 

"Having minority and women researchers in significant numbers 
and roles would increase the perspective brought to bear on 
the most difficult educational problem^ and thus increase 
. the quality and the relevance of the research. Credibility 
would be increased because, as much research indicates, we 
tend to believe those who are like us - the concept of 
homophily is articulated in both dissemination research and 
in communication research more broadly. The population of 
researchers, then, should reflect the population of the 
nation so that everyone has someone to trust! (1981, p. 4)" 

A number of barriers to the full and equal participation o f , minorities 
and women in educational research and development have been identified, 
including: 

1) inadequate quantitative and theoretical preparation (Sells, 1973). 

2) a poor record of research publication, which in turn limits 
opportunities for obtaining research funding, achieving tenure, 
and advancing in the professional field (Gappa, 1977). 

3) lack of role models and mentors (Pollard, 1977). 

- 4) lack of financial or other support for research on matters of 

concern to women/minorities (Wright, 1979). 

5) little or no access to formal or informal networks for infor- 
mation about jobs and opportunities for publication or research 
funding; perpetuation of the "old boy network" for communication 
of information (Malcolm et al., 1976). 

6) limited research assistantships (Pollard, 1977). 

7) limited experience, including lack of internships in the field 
(Artis, 1980). 

Since 1978 the NIE Minority and Women's Program has funded more than 
50 projects that have employed a variety of strategies to overcome these 
barriers, including: 

1) strong training in quantitative and theoretical research methods 
(Morrison, 1977). 

O 
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2) opportunities to apply the research methods learned in courses 
under the supervision of experienced research workers (Carter:, 
1977) . This should include grant proposal writing; conduct of 
research and evaluation projects; and preparation of technical 
reports, papers for presentation, and articles for publication 
(Morrison, 1977). 

3) opportunities to establi3h relationships with experienced persons 
who can serve as mentors (Artis, 1980). , . 

research assistantships and/or fellowships and access to .computer 
assistance, travel funds, and other forms of support for indepen- 
dent or cooperative research endeavors (Carter, 1977; Morrison, 1977) 

5) opportunities to build a sense of a research community through 
informal contacts with experienced research workers at local, 
regional, and national workshops, seminars, and conferences 
(Morrison, 1977). 

6) opportunities to engage in internships related to the career 
aspirations of the program participants (Banta, et al, 1982). 

7) advanced training opportunities through workshops or formal 
coursework (Artis, 1980). 

The models for delivery of services which have been employed in the 
Minorities and Women's Program include internships in government or research 
agencies for staff in R&D positions, fellowships in a university setting, 
R&D seminars and workshops, collaborative capacity building involving a 
major research university and a developing institution, and university- 
^onsored forums and publications designed to disseminate information 
on issues 'of interest to minorities and women. 

One of the university-based programs was carried out at the University 
of Tennessee, Knoxville. In the sections which follow, that program will 
be described and its accomplishments evaluated. 

Program at the University of Tennessee, Knoxville 

In May 1979 a three-year NIE project entitled "Experimental Program 
for Opportunities in Advanced Study and Research in Education 11 was 
initiated at the University of Tennessee, Knoxville (UTK). The program 
provided fellowship support and a variety of enrichment experiences for 
18 (six each year) minority and female second-year doctoral students. 
It was designed to assist program participants in gaining entrance to 
the "most qualified 11 applicant pool for positions in educational research 
and development. ' 

Doctoral students in the UTK fellowship program received thorough 
training in quantitative and theoretical research methods in their 
regular coursework and in specialized mini-courses provided for them by 
faculty members with expertise in areas not covered in scheduled courses. 

To supply enrichment experiences for the doctoral students, or 
Student Associates (SAs), a network was formed of UTK faculty members 
and "Field Associates," individuals holding the doctorate and employed 



in local education agencies or ,other social service organizations, B6th 
Faculty and Fiel'd Associatesj^ovided the settings and the necessary 
guidance to permit SAs to apply principles learned in coursework in 
proposal writing, project design and data collection in research and 
evaluation studies, analysis and interpretation of data, technical report 
writing, and preparation of conference papers and articles for publication 

Interaction of project personnel was promoted through structured 
exercises in group dynamics and informal gatherings in settings outside 
the University. Students traveled with Faculty and Field Associates to 
several professional meetings each year. Through these interactions as 
well as the various research groups, which were formed, a number of 
mentor relationships were fostered, and a network of project associates 
-was formed. 

In addition to fellowship funds the NIE grant made possible the 
provision of several types of assistance that promoted the ^involvement 
of project associates in educational research and development work, 
including clerical assistance for the typing and editing of manuscripts, 
computer services for the analysis of data, and funds for travel to 
conferences and workshops where papers were presented and/or specialized 
training was pursued. 

Through .contacts supplied by Faculty and Field Associates each of 
the SAs was afforded the opportunity to participate in one or more intern- 
ships that provided experiences related to her/his career goals. Settings 
for internships ranged from the offices of the Dean for Research and the 
Executive Vice-President of the University of Tennessee to an evaluation 
consulting firm in Massachusetts and the personnel benefits office of 
IBM in Armonk, New York. 

Throughout the project the SAs gained hands-on experience in 
management and evaluation of a complex project by operating the NIE- 
sponsored program themselves with guidance from the project director. 
The positions of Chair, Scribe, and Budget Monitor were rotated quarterly 
so that each SA gained experience in program planning and organization, 
budgeting, technical report writing (preparing quarterly and annual 
project reports for NIE), and program evaluation. 

Over three years a total of 45 individuals participated in the projec 
including* 18 Student Associates, 8 Field Associates, and 19 Faculty 
Associates. Of this total 38, or 84 percent, were minorities and/or 
females. All of the SAs and Field Associates were minorities and/or 
females. Twelve of 19, or 63 percent, of the Faculty Associates wfcre 
females; four, or 21 percent, were black females. 

Evaluation of the UTK Program 

Design and Instrumentation 

Bagenstos has identified a critical problem in providing summative 
evaluation of the projects sponsored by the NIE Minorities and Women's 
Program: 

The most difficult part of designing the evaluation has been 
determining appropriate outcome mensurtsH-dependent variable**. 
BocauHCi the real outcomes - sueccHf* for Individual parUcipanlN, 
removal of institutional, barrier:*, change in the. nature of 
the research agenda - are so lony, term, the need was to find 
"proxy" measures. (1981, p. 8). 
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Bagenstos has suggested that until long-term achievements can be 
assessed, formative evaluation can be undertaken by looking at details 
of project implementation an4 how those were perceived by participants; 
and summative evaluation can be initiated by investigating research . 
productivity of participants, positions obtained by persons completing- 
the project knd time and energy which they devote to research, and 
immediate impact on the institution where the project was conducted. 

As noted earlier, the \JTK fellowship students designed and carried 
out their own program evaluation for each of the three project years. 
Of necessity most of the techniques employed provided formative data, 
and emphasis was placed on individual development and self -assessment. 

Immediately following the end of the final year, the project director 
and two Student Associates developed a series of survey instruments 
to obtain additional evaluative information from all project participants. 
At that point 9 of the 18 students who had received fellowship support 
during the three years of the project had completed their degree programs, 
while 9 were continuing their studies'. 

Two questionnaires containing a core of similar items 4s well as 
questions* tailored for each audience were designed for (1) Student Associates 
and (2) Faculty and Field Associates. When Student Associates' Question- 
naires had*-been returned and analyzed, a second survey instrument for 
the students was developed and administered in order to gain additional 
insights into certain responses provided on the first instrument. The 
first questionnaire for SAs was returned by all 18 of the fellowship 
recipients, the second by 17 of 18. The survey for Faculty and Field 
Associates was sent to 19 individuals who had been associated with the 
project for all three years and was returned by 13, or 70 percent. 
The findings summarized below are baged on the combined responses to . - 

all three questionnaires. 

Results of Formative Evaluation 

Participation and perceived value of project experienc es. All 
project participants were asked to assess in a number of ways the set 
of experiences made available to SAs as part of the NIE-supported program. 
Tabl£ 1 contains a summary of SAs 1 ratings of the experiences in terms 
of (1) the extent of their own participation (scale: l«Not at all* 
5»Very Extensively) and (2) the perceived value to them (5*Excellent Value) . 
Participation in all but two of the activities was moderate to extensive, 
and in general there was a close correspondence between level of partici- 
pation and perceived value of the experiences. All experiences were viewed 
as having he&n at least of moderate value, and most. were considered of 
good to excellent value. 

SAs considered the following six experiences to have been of most 
value : 

1. Engaging in an internship 

2. Travel to present at professional meetings 

3. Travel to obtain training 

4. Developing a written report of research 

5. Effectively managing time 

6. Presenting research results orally * ^ 



TABLE 1 



Student Associates 9 Responses to Value and Extent 
of Participation in Project Activities 



Median Median 
Extent of Value of 
Participation Experience Activity 



3.5 


5 


Engaging in an internship 

> 


t 

3 


5 


Travel to present at professional 'meeting 

« 


2 


5 


Travel to obtain training 


4 


4.5 


Developing a written report of research 


4 


4.5 


Effectively managing time 


3 


4.5 


Presenting research results orally 


4 


4 


Increasing understanding of Issues re: women/minorities 


4 


• 

4 ' 


Developing interpersonal skills 


3.5 


4 


Working-with a mentor through NIE 


3 


4 


Defining a research topic 


3 


4 


Studying research design /methodology 


3 


4 


Developing research design 


3 


4 


Developing methodology for data collection 


3 


4 


Developing management skills through project management 


3 


4 


Increasing/strengthening professional networks 


3 


4 


Designing/carrying out program evaluation 


3 


4 


Using automated data processing methods 


3 


4 


Developing a grant proposal 


3 


4 


Developing leadership skills 


3 


3.5 


Selecting/developing method of analyzing data 




3 


Developing/monitoring a budget for a research project 


2 


3 


Developing skills as a professional consultant 



a 

Faculty and Field Associates also perceived experiences 1, 2f and 6 above 
to have been of most value to the SAs, but in place of experiences 3, 4, 
and 5 the experienced professionals placed the following three experiences 
\ in their llBt of the six most valued activities: 

. Participating ^in a relationship with a mentor associated with the 
project 

.Developing interpersonal skills 
.Developing an appropriate research design 

SAs, Faculty, and Field Associates agreed that developing skills 
as a professional consultant ai^d developing/monitoring a budget for a 
research project were not experiences that had beeti as significant for 
SAs as most others. * . 

The item "travel to obtain training" presented the largest discrepancy 
between extent of students' participation in an activity and its perceived 
value. Apparently, even though there were only a few opportunities ttf 
attend training workshops and conferences located outside Knoxville, SAs 
considered those to have been of great value. 

When presented with an unstructured opportunity (i.e., an open-ended 
question on the survey instrument) to identify the project activities 
they "valued most, 11 SAs provided a perspective that differed only slightly 
from that conveyed through responses based on the structured listings. 
As the content of Table 2 conveys, the most valued opportunity was that 
of applying one's research skills in projects conducted cooperatively 
with others, especially with individuals outside one^s own discipline. 
Opportunities to travel to attend professional meetings ^and/or present 
papers, and to become pirt of a professional network also were considered 
important benefits. 

Field and Faculty Associates also identified opportunities 'for 
professional travel, application of research skills in interdisciplinary 
settings, and network-building as the outstanding positive features of 
the NIE project. S f 

When Faculty and Field Associates were asked, "What project activities 
were of most value to you ?" the following experiences were listed with 
equal frequency: 

•the privilege of having the assistance of doctoral students 
and computer and clerical support as they pursued their own 
research. 

.the opportunity to obtain funding for travel to present papers 
at professional meetings. 

.the chance to engage in codperative, interdisciplinary research. 

Skill levels and perceived relationship to the project . Skills and 
abilities related to the project experiences just described were listed, 
and SAs were asked to (1) assess their own level of expertise with retipect 
to each, and (2) estimate the extent to which that level of expertise was 
related to experiences provided by the project. Level of expertise was 
to be judged by comparing oneself with fellow doctoral students or pro- 
fessional colleagues with the same level of experience. SAs rated level 

3 



TABLE 2 

Activities Identified as "Most Valued" by Student Associates 

- 0 * < ' 

% of Respondents 

Listing Response ' * Re sponse 

56 Opportunity to participate in research projects at varying 

stages of development with people outside one's own department 

29, Travel to attend and/or present at professional meetings 

17 Becoming part of professional networks * 

17 Opportunity to meet and work with graduate students from 

otfer colleges and/or disciplines 

11 Colloquia and workshops 

11 Internships' 

11 Increased awareness of women's /minority issues 

f 

11 Chance to work with/ learn from mentors 

6 The variety 'of experiences available 

6 Project management 
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of expertise on a 5-Rpint scale with 1-poor or nonexistent and 5-outstapding; 
relationship of skill level to project experience was rated on a 5-point 
scale with 1-not at all related and 5-completely related. Students' median 
responses are summarized in Table 3. 

The range of median responses on both scales is quite narrow: The 
perceived level of expertise on every skill was rated 3(average) or 4 '(above 
average), and perceived^ relationship to project participation was 3 (moder- 
ately related) or 4 (highly^lated) for all but three of the skills. These 
response levels seem to indicate that SAs considered their skill olevels 
with respect to 19 project-related activities to be relatively high and 
attributed to the NIE project a moderate to high proportion of the experience 
that had led to the development of those levels of expertise. 

Witk respect to v the five' skills/abilities listed below SAs considered 
their level of expertise to be above average (median* rating of 4) and the 
relationship of that expertise to the project, experience to-be high (median 
rating of 4). 

1. Ability to engage effectively in an internsh ip. 

2. Ability to develop a written report of research results. 

3. Ability to work effectively with a mentor. 

4. Skill in increasihg/strengthening professional network ties. 

5. Increasing understanding of issues related to ttomen and minorities. 

Apparently SAs attributed the development of their skills in using 
automated data processing, defining a research topic, and performing the 
role of professional consultant more* to experiences gained in connection 
with their academic programs than to experiences provided by the NIE- 
supported program. / 

Value o£ experiences and their relationship to the project . Combining 
responses concerning (1) the value of project experiences and (2) the 
relationship to the NIE prpjectf^f the development of skills connected 
with these experiences produced a list of activities that were both highly 
valued (median rating of 4-5) and seemed to involve skills that were promoted 
strongly (median rating of 4) by the NIE program. Apparently the following 
four experiences were for students the most valued benefits of the project 
when it was compared to experiences provided in the traditional doctoral 
program: 

1. Engaging in an internship.* 

2. Developing a written report of research. 

3. Increasing and strengthening professional networks. 

4. Increasing understanding of issues relating to women and minorities. 

When the response format was not specified, and SAs had the opportunity 
to construct their own list of experiences that differentiated the NIE 
project experience from that of the typical doctoral program, a somewhat 



TABLE 3 



• Level of SA Expertise and 
Relation of That Level to the Project Experience 

Median Response Median Response to 
to Level of Relationship to 
Expertise NIE Experience Skill/Ability - 

4 4 Ability to engage effectively in an internship 

4 4 Ability tp develop a written report of research 

v v results 

4 . 4 ^ Skill in increasing/strengthening professional 

network ties 

4 4 Increased understanding of issues re: women/ 

minorities - 
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Ability to work effectively w^th a mentor 



4 3 Ability to effectively manage time 

4 m 3 Skill in presenting research results at 

professional meetings 

3.5 3,5 Skill in presenting* research results orally 

3 3.5 Skill in designing/carrying out program evaluation 

3 3 Skill in research design/inethodology 

3 3 Skill in developing appropriate research designs 

3 3 Skill in developing appropriate data collection 

techniques 

3 3 Skill in managing a project such as NIE 

3 3 Skill in developing a -rant proposal 

3 3 Ability to select appropriate methods of data 

analysis 

3 3 Ability to develop/monitor a project budget 

3 2.5 Skill in defining a research topic 

3 2.5 Ability to perform the role of professional 

consultant 

3 2 Skill iti using automated data processing 



different perspective emerged. As the content of Table 4 indicates, 
students were most favorably impressed with their opportunities to apply 
a variety of research skills to a variety of research problems in settings 
where their contributions were valued by established professionals. 
Professional travel, internships, and mentor relationships also distin- 
guished the project experience from that of the doctoral program. 

Faculty and Field Associates also were given an opportunity to 
construct their own responses to the question of differential experiences 
provided by the NIE project. These individuals most frequently identified 
opportunities for students Lo (1) obtain financial support for travel and 
other research-related activities, and (2) establish professional and 
personal relationships with established professionals in addition to those 
established >vjth members, .of the doctoral committee^— — V. 

All participants were asked to identify those project experiences 
which were ofyleast value to them. This item produced few responses and 
no clear trexiAs. Four of the 18 SAs said their internship experiences 
had not met all the expectations they had established for them. . Three 
students mentioned individual colloquium topics that had not appealed to 
them. Two students said project social activities had been the least 
valuable aspect of the project. No^two Faculty/Field Associates mentioned 
the same activity as being least valuable. 

The mentor relationship . On the second questionnaire for SAs, 
which was designed to provide more detailed information than the * first 
concerning certain aspects of the project, the students were asked to 
describe their perceptions of a mentor. These perceptions seemed to 
cluster in three categories: 

(1) The "interested party 11 role model who does not participate 
actively in decision-making (10 respondents); 

(2) The "experienced comrade 11 who gives advice and takes a 
relatively active role in guiding the novice (4 respondents); 
and 

(3) The "instrumental 11 mentor who acts N as a ground-breaker, 

opens doors, and actively gives career guidance (3 respondents) ♦ 

Twelve, or , two- thirds, of the SAs said that they had had a mentor 
as a result of their involvement in the NIE project. Eight of thirteen, 
or 62 percent, of the Field and Faculty Associates said they had served 
as mentor for one or more SAs during the course of the project. 

All SAs who had experienced a relationship with a mentor considered 
that experience Vextremely" or "somewhat" positive (rating of 5 or 4 on 
a 5-point scale). When asked how the NIE experience had enhanced their 
opportunity for a mentor experience, 37% responded that the project structure 
encouraged such relationships, 22% said it facilitated such relationships 
with persons outside their base departments, and 11% said it provided an 
opportunity for skill development. Responses are summarized in Table 5. 

Benefits which SAs identified With mentor relationships (see Table 6) 
paralleled their definitions of mentors, i.e., mentors were seen as 
sources of technical knowledge and professional as well as personal support, 
faculty .and Field Associates responding to a similar item said "a mentor 
can open doors and provide professional contacts and opportunities" and 
"the student can benefit from the mentor's past mistakes." 
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TABLE 4 

Activities Differentiating the NIE Experience from a Typical Doctoral Program 



% of SAs 

Giving Response Response 

50 Chance to work on many different research tasks and projects 

as a valued contributor 

44 Chance to work with faculty and students from many different 

departments and colleges 

44 Chance to travel to and/or present at professional meetings _. 

19 Chance to intern in self-selected settings with Project 
support 

19 Chance to work with mentors outside one's own department 

11 Chance to participate/present in colloquia and mini-courses 

11 Chance to learn more about women's/minority issues 

6 Chance to participate in workshops 

6 Help in establishing/expanding a professional network 

6 Access to specialized consultants 



u 



TABLE 5 



Student Associates' Perceptions of 
Ways in Which the NIE Experience Enhanced Mentor Relationships 



% Response 

37 Project structure encouraged such relationships 

22 Encouraged such relationships with persons outside one's 

own base department 

11 Provided opportunity for skill development 

6 Internships developed into mentor relationships 

6 Helped SAs learn how to find a mentor 

6 Helped SAs learn how to be a mentor 

> 6 Provided role models 
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TABLE 6 



Student Associates' Perceptions of Benefits 
Associated with a Mentor Relationship 



44 
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% response ; Response 

44 



Mentor is a source of knowledge 
Mentor is a source of technical support 
2 ? Mentor can enhance networking opportunities 

Mentor is a source of emotional support 
Mentor can provide a role model 
Mentor can aid in job placement 
Mentor can provide research opportunities 
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Networking . SAs were asked to provide their own definition of a 
professional network. Most respondents viewed networks as sources of 
infonriation. Others considered them to be more actively f acilitative, 
i.e., sources of job or research opportunities, as in an "old boy network/ 1 
About half of the respondents thought that project participation had 
allowed them to develop new network ties, whereas about one- third thought 
that NIE participation had helped strengthen existing ties. The SAs said 
network development had been enhanced through (1) travel to professional 
meetings or for training, (2) relationships with other students, and 
(3) opportunities to work with faculty. 

Group research . The opportunity to engage in research activities 
with others was a principal project benefit, according to students as 
well as Faculty and Field Associates. When asked if they had participated 
in group research with other project participants, 70 percent of the field/ 
faculty respondents and 88 percent of the SAs replied affirmatively. 
However, approximately half of the students and 40 percent of the 
established professionals said they felt their productivity was greater 
when they worked alone than when they were part of a group. 

In response to the question, "What research tasks are best accomplished 
in groups?" SAs most frequently mentioned, "developing papers for presen- 
tation/publication," "data collection," "data analysis," and "evaluation." 
Faculty and Field Associates said, "developing the design for a research 
project" and "evaluation." 

Awareness of minorities 1 and women's issues . The SAs were asked to 
list ways in which the NIE project had enhanced their awareness of issues 
relating to women and minorities. Most mentioned the colloquia as 
important sources of information about these issues. (Colloquium topics 
were selected by SAs and presentations were invited from faculty and 
outside consultants.) Approximately half of the SAs mentioned informal 
group discussion as a source of information, and one-fourth mentioned 
workshops or conferences. Internships, research involvements, and the 
SAs 1 own research also were mentioned. Faculty and Field Associates also 
considered the colloquia to be important sources of information about 
issues of interest to minorities and women, and mentioned as another source 
the contacts with minorities and women who participated in the project. 

SAs were asked how their increased knowledge of issues of concern 
to minorities and women would influence them professionally. Forty percent 
said they had acquired new ideas for solving related problems, and had 
experienced an increase in their willingness to provide support for other 
minorities and women. Sixteen percent noted .that their own behavior had 
been changed, i.e., they had become more assertive, less passively accepting 
of inequities. Several of the students said they had become active 
politically or had changed career goals. 

Res ults of Summative Evaluation 

Effects of p roject participation on employability of Student 
Asgoclates. An important objective-of the NIE project at UTK was to 
incrcin&e the likelihood that participants would be classified in the "most 
qualified applicant" pool when they applied for employment. While only 



seven of the SAs had obtained full-time employment (4 as faculty in higher 
education, 2 in industrial training) at the time the follow-up questionnaires 
were administered, 16 of 18, or 87% of the respondents believed project 
participation had significantly enhanced their employability. Sixty-two 
percent of the Faculty and Field Associates concurred in this assessment. 

SAs were asked to describe the effects of project participation on 
the likelihood of their being placed in the "most qualified applicant" 
pool. Their responses indicated that the research experience was 
valued by prospective employers. In some cases the project experience 
itself, especially the internship, led to part- or full-time employment. 

Research utilization . The real pay-off of a project such as that . 
supported by NIE at UTK may be the degree to which participants increase 
their use of, or participation in, educational research. To assess how 
SAs were affected by the project in this sense, SAs were asked to respond 
to four questions: 

1. How has the project influenced your ability to participate 
in/benefit from educational research? 

2. To what extent are you currently using the results of 
educational research? 

3. To what extent are you currently engaged in educational 
research? 

4. What are your plans for future research involvements? 

Most SAs stated that the. research experience gained during the 
project had influenced their ability to participate in or benefit from 
educational research by enhancing the development of their research skills. 
Other comments indicated that students, had achieved an increased aware- 
ness of research issues, increased ability to critically analyze disseminated 
research, new skills in the technical aspects of research dissemination 
and new skills in the solicitation of appropriate assistance ("asking the 
right questions"). When asked how they currently were using the results 
of educational research, four SAs responded that they were using results 
as student consumers, three as classroom instructors, and one each as a 
critical assessor of disseminated research, as a program evaluator, as 
a data collector, and in developing a model for application in industrial 
on-the-job training. 

SAs described their current research involvements as work on a 
dissertation (37%), continuation of projects begun through NIE partici- 
pation (37%), data collection as a part of employment (11%), writing 
position papers (6%), and developing industrial models (6%). Only one 
student stated that her involvement in research was limited or non-existent. 

Projections for future involvement in educational research also were 
solicited. Only one respondent replied that her involvement was likely 
to be negligible. Two SAs were largely concerned with their dissertations. 
Others mentioned specific research interests in varying stages of develop- 
ment, including maternal satisfaction in home vs. professional settings, 
black children in the community, and women in management, among many others. 
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Research production , A project which attempts to increase the research 
participation of women and minorities also may be evaluated on the basis 
of the products — research projects and disseminated research — which 
it generates. In this regard, the NIE project at UTK was particularly 
successful. The data in Table 7 indicate that on the average each year of 
the project each of the six current SAs was involved in at least three 
research endeavors that resulted in a written 'product. The average number 
of paper presentations alone was three for each SA during Year 1 and Year 2, 
and declined only slightly in Year 3. The .number of publications had 
increased five-fold by the end of the third year. The number of proposals 
developed also increased dramatically as the project became established. 

Impact on barriers to recognition of research contributions by 
minorities and women . The NIE project on the UTK campus heightened aware- 
ness of the benefits of involving minorities and women in educational 
research and administration. 

Several SAs ^fere selected for special awards/recognition (see Table 8). 
One Faculty Associate was selected for participation in a research symposium 
for minorities as a direct result of the NIE project. The project director 
received the 1980 College of Education Faculty Research Award and a Chan- 
cellor's Citation for Extraordinary Service to. the University in 1982. 

The appointment of the first full-time female faculty member in one 
of the departments of the College of Education at UTK can be attributed 
directly to the effective operation of the network of pt'Dject participants. 

All Faculty and Field Associates responding to the end-of-project 
survey indicated that they considered the project to be of such value 
that they would want to continue to participate in similar experiences 
with doctoral students even without the funding for research support 
services which was provided by NIE. 

A program incorporating the kinds of enrichment experiences which 
were provided for students by the NIE project was considered by the Dean 
for Graduate Studies and the Dean for Research to be important enough to 
continue. Participants were to be recipients of non-service fellowships 
at UTK. 



TABLE 7 



Research Production by Student Associates 
During Each of Three Project Years 



Proposals 



Year 1 



Presentations 



Publications 



18 



Year 2 



18 
9 
21 



Year 3 



16 
10 
18 



TABLE 8 



Awards and Special Recognition Accorded Student Associates 



Chancellor's Citation (Outstanding Graduate Student) 
UTK Women of Achievement Award 

2 selected to attend AERA-NIE Annual Meeting Presessions 

2 Fellowships at the Appal^Jiia Education Lab 

PDK International Committee on Public Confidence in Public 
Education 

College of Home Economics Excellence in Research Award 
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Summary 



Participants — Faculty, Field and Student Associates — in the UTK 
pre-doctoral program for mxnorities and women valued all of the professional 
experiences afforded by the project, but seemed most appreciative of the 
impetus which the project gave to bringing together professionals and pre- 
professionals - from various disciplines to (1) consider and carry out 
joint research endeavors, (2) form networks, and (3) establish bases for 
internships and mentor relationships. 

Students who received fellowship support believed that the project 
experience had been most effective in increasing their abilities to engage 
in an internship, work with a mentor, strengthen network ties, and write 
a research report. They also felt that the experience had increased their 
understanding of issues of concern to minorities and women. 

Opportunities afforded by NIE support which most clearly distinguished 
the project experience from the typical doctoral program were identified 
by participants as those involving interdisciplinary contacts, interfiships, 
technical writing practice, support for research services and travel, and 
exposure to a range of issues of interest to minorities and women. 

Almost all of the students and more than 60 percent of the professional 
participants believed the project experience enhanced the employability of 
Student Associates. Several measures indicated that both the utilization 
and production of research by participants Increased as a result of the 
NIE project. 

' Heightened awareness of the unique contributions which minorities and 
women can make to educational research and administration was demonstrated 
by the number and kinds of awards and special recognition which were accorded • 
NIE project participants over the three years of its operation at UTK. 



The results of formative evaluation of a three-year project carried 
out at the University of Tennessee, Knoxville with funding from NIE have 
demonstrated the effectiveness of a model for increasing the participation 
of minorities and women in educational research and development which 



1) strong training in quantitative and theoretical research methods, j 



2) opportunities to apply research techniques under the supervision 
of experienced professionals, 

3) opportunities to establish relationships with mentors, 

4) fellowship assistance and financial support for the conduct of 
research and dissemination of findings, 

5) opportunities to build and/or strengthen netv/brk ties through 
local associations and professional travel, 

6) internship opportunities, and 

7) advanced training in formal courses ur workshops. 



Conclusions 



includes such components as: 




ERJC 
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Although opportunities to enhance technical skills through courses 
offered by the University and mini-courses or colloquia conducted under 
project auspices were readily available to each project participant, the 
acquisition of quantitative and theoretical expertise was not valued as 
highly by UTK participants as were opportunities to apply knowledge and 
to increase professional contacts. Interdisciplinary research experience, 
networking, internships, and relationships with mentors consistently 
received the highest ratings when a series of measures was utilized to 
evaluate the effectiveness of the components of the training model. 

Since the action of convening individuals with varying levels of 
expertise to work cooperatively on problems of common interest can be 
accomplished without external funding, most participants said they were 
willing to continue to be involved in activities associated with the project 
even after the NIE grant was terminated. Unfortunately, however, without 
the benefits provided by the external funds, e.g., financial assistance 
for students, released time for a faculty coordinator, reimbursement for 
professional travel, clerical and computer support, the impetus for contin- 
uation of such a training program must come from the energy and good will 
of volunteers — not usually a very reliable source. 

The visibility and perceived success of the NIE program for pre- 
doctoral fellows at UTK assured its continuation in some form. However, 
the absence of funding for coordination and support services has reduced 
the scope, therefore the impact, of the continuation effort. 

Satisfactory summative evaluation of the UTK project must await the 
establishment of program graduates in their professional careers. Never- 
theless, already there is some evidence that the social context of education 
will be influenced by increased awareness of the contributions that can 
be made by minorities and women associated with the NIE program. Ideally 
the research productivity of NIE pre-doctoral fellows would be contrasted 
with that of a comparison group, but the number of minority doctoral students 
in educational fields at UTK was so small that most were selected to 
participate in the program. Informal comparisons indicated, however, that 
the number of research projects planned, in progress, and completed by NIE 
fellows exceeded that of comparable numbers of their peers in traditional 
doctoral programs at UTK. In addition, the predominantly white male 
faculties of several departments in the College of Education at UTK acquired 
new awareness of minorities' and women's issues by serving as research 
group leaders and as mentors for project participants. Evidence of their 
increased appreciation of the capabilities of minorities and women is 
provided by the numbers and kinds of special awards and recognition which 
they accorded to the pre-doctoral fellows and some female faculty participants. 
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